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Multimedia Mail Meeting Notes 


Status of this Memo 


This memo is a report on a meeting about the experimental multimedia 
mail system (and in a sense a status report on that experiment). 
Distribution of this memo is unlimited. 


1. Introduction 


A meeting was held at Bolt Beranek and Newman on 23-24 July 1984 to 
discuss recent progress by groups who are building multimedia mail 
systems and to discuss a variety of issues related to the further 
development of multimedia systems. Representatives were present from 
BBN, ISI, SRI and Linkabit. The list of attendees appears at the end 
of this note. 


The result of this meeting is a series of agreements that will be 
incorporated in the next set of experiments with multimedia mail as 
well as a set of items for further action. 


Note: There are references in this document to notes in a series 
devoted to multimedia mail. These notes are available on-line in the 
directory [USC-ISIF]<MMM> and have the names MMM-N.TXT where N is the 
note number. The file MMM-INDEX.TXT is a list of all of the notes in 
the series. These public files may be copied via FTP using the FTP 
username ANONYMOUS and password GUEST. 


2. Review of Status 


Status reports on work accomplished in the last year were given by 
each organization. 


2.1. BBN 


The initial implementation of Diamond is complete and runs on the 
Jericho workstation. Diamond currently supports the exchange of 
compound documents which contain text, graphics, images, voice and 
spreadsheet/charts. A demonstration of this system was presented 
showing both the user’s view of Diamond messages and message 
management as well as the interactions between the components of this 
distributed system. Diamond currently uses the TOPS-20 implementation 
of MPM for inter-cluster message transport but the plan is to 
integrate an implementation of MPM for the Sun Workstation into 
Diamond. Current activity is focused on porting Diamond to the Sun 
Workstation. A first version of Diamond for the Sun is nearly 
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completed and parts (the document editor) were demonstrated running 
on the Sun. Diamond will be used in the ADDCOMPE testbed with 
100-200 users expected in the next year or so. Future plans include 
building on the experience gained with Diamond in the area of 
multimedia conferencing, extending the use of multimedia into other 
application areas and applying the distributed architecture of 
Diamond to other application areas. 


Ze2e) St 


A new effort aimed at developing a user interface on a Xerox 1108 
(Dandelion) workstation has just begun. All of the implementation is 
being done in Interlisp. Initial work has been done to implement IP 
and TFTP on the 1108 as well as a document editor that makes use of 
the Interlisp-D window system. Work on the user interface that was 
developed on the Perg will be cycling down. The implementation of 
the MPM on TOPS-20 is essentially complete with the addition of MPM 
to SMTP mail conversion; no major changes are anticipated. The 
TOPS-20 MPM will be used as the message transport facility for the 
1108 user interface implementation. TFTP will be used to get 
messages from the 1108 to the TOPS-20. 


2i 3s SRI 


The SRI multimedia mail system consists of three parts: The 
Multimedia Mail Handler (MMH) which is the user’s interface for 
managing mail, the Structure Editor (SE) which is used to view and 
compose multimedia messages and the MPM for mail transport. This 
system is implemented on the Sun Workstation. The first release of 
the MPM on the Sun will be ready for distribution at the end of this 
summer. The SE is used to view and compose structures of multimedia 
objects. Currently there is support for text, voice and graphics. 


Another effort at SRI involves integration of applications to run in 
the ADDCOMPE testbed. Diamond will be the first example of an 
application which uses multimedia data in the testbed. SRI is 
interested in examining the issues associated with multimedia systems 
to determine how multimedia data can be used in other applications 
that might be put into the testbed. 


2.4. Linkabit 


Linkabit has recently received a contract to work on protocol 
evaluation, problems associated with working in a large internet 
environment, and new real-time end-to-end services. They will be 
working with Sun workstations. Areas of interest are protocols, 
multimedia conferencing and domains. 
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3. Discussions and Agreements 
3.1. Conversion to the New Multimedia Syntax 


There was general agreement that in future experiments we would all 
adopt the revised syntax for multimedia mail as described in the 
Final Report to SRI Project 5363. It was agreed that RFC767 should 
be revised to reflect these changes. The timing for switching over 
should be as soon as possible and should be completed by October 1, 
1984. 


3.2. Graphics Representation 


A wide ranging discussion on the way in which graphics is to be 
represented in multimedia documents occurred. It was generally 
agreed that the first style of graphical object to be included in 
multimedia messages would be a simple line-drawing, such as those 
that can be produced by the many "draw" programs (e.g. LisaDraw) 
currently in existence. Attention was focused on the two existing 
standards (ACM-CORE and GKS) and the interim protocol used in the 
Diamond system. Two major problems with the existing standards were 
mentioned: 


o In both ACM-CORE and GKS grouping is inadequate or non-existent. 
This means that it is difficult or impossible to build up a 
composition of several graphical objects and then manipulate 
that composite as a single graphical object. 


o Neither ACM-CORE or GKS have specified a standard method for 
representing graphical drawings in memory (e.g. long term file 
storage). This is one of the most important aspects of a 
graphical standard for multimedia mail. The focus of graphical 
standards so far has been towards driving devices ina 
independent manner, not storing graphics in a standard 
representation. 


A presentation of the representation for graphical objects in Diamond 
was given. The protocol is documented in MMM-18 and MMM-23. 

Requests for hardcopies of the diagrams in those documents (sigh) can 
be sent to Travers@BBN. 


The discussion then focused on the level of effort required to switch 
from one representation to another. It was generally agreed that 
compared to the entire editor used to manipulate graphical objects 
(e.g., the "draw" program), the part that reads or writes objects 
from or to files is relatively simple. Most draw programs have a 
unique internal representation which is built when reading the file 
representation and used as the source when writing the file 
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representation. It is this relatively small portion of a graphics 
editor which is impacted by switching from one file representation to 
another. Thus it seemed that the approach of adopting one interim 
representation now and planning to switch to a standard 
representation when work on the standards solve the problems noted 
above was reasonable. 


After considerable examination of the issues, the following decisions 
were reached: 


1. The representation for graphics used in Diamond and documented 
in MMM-18 and MMM-23 will be adopted as an interim 
representation for graphics in multimedia mail. It will be 
known as the MMGraphics1l protocol. 


2. We will actively track development of the GKS standard and also 
examine a GKS-subset that has been developed by Sandia Labs. 


3. We agreed to settle on an adopted international standard 
eventually. 


3.3. Document Presentation Semantics 


There was a presentation of the ideas contained in MMM-22: "A Format 
for the Construction of Multimedia Messages". The resulting 
discussion addressed the following issues: 


o Presentation of documents on display devices with different 
characteristics (dimensions, resolutions, available fonts, 
etc.). 


The essence of the conversation was that there is no single set 
of fonts, or page sizes that will cover all of the 
possibilities. There was a strong feeling that as long as the 
display surface was of reasonable size that a document should 
be presented in a "correctly" formatted manner. Rather than 
the originator of a document specifying hard page boundaries, 
the intent of the originator regarding formatting and grouping 
of objects in the document should be preserved and used when 
the document is actually presented on a display device. A 
window on a bitmap display and a hardcopy page printer are both 
examples of display devices. 


o The desire to represent the kinds of documents that currently 
exist in the world of hardcopy as well as to represent documents 
that can take advantage of the new possibilities of electronic 
creation, storage and presentation. 
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Several points were made: 


{x 


One of the first goals for multimedia systems should be to 
represent the types of documents that are prevalent in the 
hardcopy world. People are already familiar with these 
documents and will expect multimedia systems to, at least, be 
able to deal with them. 


In an effort to provide the capabilities of electronically 
originated documents based on the hardcopy model of documents, 
we should not eliminate the great potential of electronic 
documents that have much greater reactive qualities. For 
example, a document where a graphical figure and a textual 
explanation of that figure are linked so that as long as the 
explanation is being read the figure is visible. 


In many situations being able to carry away a paper copy of a 
document is a requirement even if the document was not 
primarily intended to be presented in hardcopy. 


The following agreements were made: 


Ie 


Forsdick 


A method for recording the author's intent regarding the 
presentation of a document should be developed. This 
representation would defer decisions on final presentation 
bindings of size, resolution and fonts to the reader's document 
presenter. 


Topics addressed by this representation will include: 

o Grouping of objects 

o Limited Font attributes (e.g., normal, bold, italic) 

o Headings, Footings 

o Sectioning 
Of course the reader’s document presenter is free to ignore any 
of the author’s intentions it cannot deal with. The point of 
this representation is to record the author’s desires but to 
defer final decisions on how to use the intentions until the 
capabilities of the reader are known. 
This representation will lie some where between the rather 


loose spatial and temporal positioning commands currently in 
the protocol (Sequential, Simultaneous and Independent) and the 
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absolute positioning of a system that defines rigid page 
boundaries and absolute positions for object placement on a 


page. 


2. We will NOT try to make this representation handle all of the 
issues addressed by modern document formatting systems. 
Instead we will skim off some of the most useful ideas but 
perhaps limit the flexibility present in these complex 
formatting systems. 


3. The document representation will be able to describe 
relationships between objects that make use of the capabilities 
of electronic document presentation, such as simultaneous 
presentation (i.e., two objects which are visible at the same 
time) and overlay presentation (i.e., two (possibly 
transparent) objects which occupy the same area in a document, 
which may also be separated under viewer control). 


4. Methods should be developed for all aspects of the document 
representation for presenting the document in a hardcopy form. 
This applies both electronic documents that fit the tradition 
hardcopy model as well as those that make use of the more 
reactive features of the representation. 


3.4. Directory Service 


There is an increasing need to be able to determine attributes of 
users, hosts and domains throughout the DARPA Internet. For example, 
when composing the header fields of a message it is useful to be able 
to inquire about the mail box location of a person to whom the 
message is addressed. Likewise, there is need to determine the 
services provided by a host so that requests that will never be 
satisfied can be avoided. 


The feeling of the group was that work on the Internet Domain system 
(being done at ISI and Berkeley) would answer some of these problems 
and that we should examine the design documents to see how that 


system might help us (see RFCs 882 and 883). The WhoIs server is 
useful, but only for information about the text mail box of a person 
(see RFC812). 
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3.5. New Media Types 


The discussion dealt with three topics: A proposal for a new media 
type, ideas for other new media types and provisions for dealing with 
unknown media types. 


A description of the Diamond SpreadSheet/Chart media type was 
presented. This is documented in MMM-24. In this media it is 
possible to represent a table containing numbers, labels, dates and 
formulas. A unique attribute of this media type is that the 
spreadsheet model as well as the data are transmitted. The reader of 
a document containing a spreadsheet object can test what effect 
different data would have on conclusions suggested by the spreadsheet 
object. A spreadsheet may appear as a table and/or one of several 
alternative business charts (line graph, scatter graph, bar chart or 
pie chart). Rulings may be added to the tabular representation so 
that it is possible to achieve the appearance of sophisticated 
tabular data presentation. During the discussion, the point was made 
that a minimal implementation of the spreadsheet object could ignore 
the formulas and just present the values of the cells, thus allowing 
a minimal presentation of the tabular and chart information. 


Ideas for new media types included: 
Form 


A set of fields which are Name-Value pairs. Forms can be used 
for presentation and/or acceptance of information. The act of 
filling out a form might be used (under user approval) to 
trigger sending the completed form to the appropriate person 
who handles such forms. 


Animated Graphics 


A line drawing that has temporal information encoded in the 
presentation of its components. The idea is that parts of a 
graphics object could move about the object during its 
presentation. For example, an arrow could move about a map 
showing a route to be followed. There was some discussion 
about how this would interact with other media. For example, 
how could an arrow moving about a map be coordinated with voice 
instructions on how to get from one place to another. There 
were no decisions about how best to accomplish this. 


Finally, we agreed that all of our systems should be prepared to 
accept (and possibly ignore) media types that are not currently 
implemented. The common way of dealing with this is to include a 
statement of the form "An object of type <Type> appears here". With 
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the regularized syntax that has been adopted many of the common 
attributes of all object types will be able to be understood but the 
actual type may not be implemented. In Diamond we would like to use 
the MPM to transfer Diamond messages between Diamond and non-Diamond 
clusters. Currently if we were to include a spreadsheet in one of 
these messages, all of the other implementations of multimedia mail 
would probably end in the debugger when they went to process our 
messages, rather than indicate that there was something that they 
didn’t quite understand. 


3.6. MPM Support 


By the end of the summer there will be two implementation of the MPM: 
on TOPS-20 and on the Sun Workstation. We agreed to try to set up 
the following operational MPMs: 


Organization Host MPM Implementation 
ISI ISIF TOPS-20 

ISI ISIB TOPS-20 

SRI ? Sun Workstation 
BBN ? Sun Workstation 
DARPA ? Sun Workstation 
Linkabit DCN6 Sun Workstation 


The idea behind this agreement is to get wide geographic coverage to 
allow us to use multimedia mail on a regular basis and to test the 
impact of realistic use of multiple communicating MPMs using the 
Internet. 


3.7. Floating Point Data Type 


In the representation for data defined in RFC759, there is no way to 
represent floating point numbers. We agreed that a new data type 
should be added, called Float64 which is the 64-bit IEEE standard 
floating point number representation. 


3.8. Captions 


The idea of including a text caption as an optional property of every 
object was discussed. There are several uses of such a caption: 


o For media like voice which do not have an implicit visual 
representation, it is useful to include a caption indicating 
something about the object. This caption can serve as a visual 
indication of the presence of the non-visual object. 
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o When an implementation of a multimedia message system doesn’t 
support a given media type, it can be useful to give some 
information about the object in the form of a text passage. 


o In some situations, it is important to present an outline of a 
document. Captions associated with each object could be used to 
generate a shortened abstract of the document. 


We agreed to add to all object types an optional property whose name 
is "Caption" and whose value is of type Text String. 


3.9. More Users of Multimedia Mail 


We need to increase the use of multimedia mail to gain more 
experience with issues that need attention. This can be done by: 


o Encouraging more sites to participate in the experiments. There 
are several possible sites which have Sun workstations that 
could be configured to run an MPM and one of the multimedia 
message systems. 


o Making the MPMs perform translations to and from SMTP text-only 
mail. At BBN, the Diamond Import/Export component performs 
translations in both directions and this has proved very useful 
in testing the operation of our system. In addition, the 
inclusion of statements such as <Graphics appears here> might 
spark interest from text-only mail recipients, although care 
should be taken not to offend anybody with this kind of "class 
differentiation". 


To the extent possible, the Sun Workstation MPM will be modified to 
perform translations to and from SMTP mail. The TOPS-20 MPM already 
does the translation from multimedia mail to text-only mail. It may 
be possible to add translation in the other direction. 


3.10. Multimedia Exploder Mailing List 


A mailing list devoted to Multimedia Mail will be set up at ISI. 
This will be of the "exploding" variety so that sending a message to 
the list will cause everybody on the list to receive a copy. To get 
on or off the list send a note to MMM-People-Request@USC-ISIF.ARPA. 
The exploder mailbox is MMM-People@USC-ISIF.ARPA. 
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3.11. Next Experiment 


The next experiment will be in January 1985. At that time we will 
try to demonstrate the following new features: 


o Use of the revised multimedia syntax described in section 3.1. 


o Inclusion of Graphics objects, in addition to Text, Images and 
Voice. 


o Use of the, as yet unspecified, document presentation semantics 
described in section 3.3. 


o Use of the Sun Workstation MPMs. 
4. Further Actions 


Several of the agreements reached require further action. I have 
added dates which seem reasonable. 


Revision of RFC759 to include Float64 data type. 
Person: Greg Finn and Jon Postel. 
Due Date: 1 September 84. 


Conversion to the new Multimedia Syntax 
Person: All groups. 
Due Date: 1 September 84. 


Revision of RFC767 to reflect revised Multimedia Syntax and 
optional Caption property 

Person: Jose Garcia-Luna and Jon Postel 

Due Date: 1 October 84. 


Specification of Document Presentation Semantics (Section 3.3) 
Person: Harry Forsdick 
Due Date: 1 October 84. 


Acquisition of GKS and GKS-subset documentation 
Person: Lou Schreier 
Due Date: 1 September 84 


Completion of initial implementation of Sun Workstation MPM 
Person: Andy Poggio 
Due Date: 15 September 84 


Multimedia Exploder Mailing List 


Person: Greg Finn 
Due Date: 15 August 84 < COMPLETED > 
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Addition of MPM<==>SMTP translation logic to Sun Workstation MPM 
Person: Mike O’Connor 
Due Date: 1 November 84 


Demonstrate Text-—Graphics-Image-Voice Document Exchange 
Person: All 


Due Date: January 85 


5. Attendees 


Harry Forsdick BBN Forsdick@BBN (617) 497-3638 
David L. Mills Linkabit Mills@ISID (703) 734-9000 
Louis Schreier SRI Schreier@SRI-SPAM (415) 326-6200 
Philip Au SRI Psa@SRI-SPAM (415) 326-6200 
Greg Finn ISI Finn@ISIF (213) 822-1511 
Mike O’Connor Linkabit OConnor@DCN9 (703) 734-9000 
Ray Tomlinson BBN Tomlinson@BBN (617) 497-3363 
Ginny Travers BBN Travers@BBN (617) 497-2647 
Terry Crowley BBN TCrowley@BBN (617) 497-2677 
Andy Poggio SRI Poggio@SRI-TSC (415) 859-5094 
Jose Garcia-Luna SRI Garcia@SRI-TSC (415) 859-5647 
George Robertson BBN GRobertson@BBN (617) 497-3632 
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